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1 Introduction 



touch screen dSs^eS ' ** apPr ° aCh 0311 be for 

indivMual E»d«SS 46,1 ^ ° n to identif y mi ^oss the 
Kamba et anwi^f- ** B 'f*" £ P °A text input problem as demonstrated in 

„ riffl^fi tSffi " dM ^ ui (•?«?■ p M.y, I suggest a novel 

• oTselS^; 1 I T af .*° kyers withta * re »°n of the keyboard Sn 

c^LrS°rSrS s b r , , SlmPle ~ ,huS ^biguating between. 
adeoLte degS of ZS" \ laIg £ PoP^on of control elements with ah 
element* or TSZ&ggZ"' ** *» the s i2 e of control 

2 Problem Domain 

3 Background 

3.1 Flag-In Keyboards or Built la Keyboards 

tex^ snSl*? .^"joern to offer a solution to the problem of easily entering 



3.2 Clip on Thumb Keyboard & Overlays 

^?« kfl t° a 5 d8 « inV S6em t0 prOYide a mable text en *y facility for small devices 
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ttS! I e -° 1 1P ,°\ 1S fee ° Verlay Aboard. Tliough these do not increase the 
SS5 h I deV1Ce ' *7 do have implications. The overlay is essentially no 

£c£^w° m 3 S ° ft f yb °? d (diSCUSSed below >' actuall y is a very expensive 
on! ^2?J? ennanently renders'the utility of a portion of the display for text input 
only, restricting the use of an already limited resource. 

33 Soft keyboards or Dialogue Driven Input Approaches. 

The soft keyboard is not really too different from the clip-on keyboard, except it is 
keyboard has the added hindrance of consuming screen display area, as does the 

ztsszSF^ soft keyboard does peimit * e — to 55 

3.4 Standard and Full Screen Keyboards 

r?o e o^ ft iS yb r d I^ m f t0 be &e m0St commonl y accepted solution; see Kamba et al 
zZfXlt tT d TUrk - (2002) ' MacKenzie ^ Soukoreff (1999) MacKenzie and 
\ W Tf' -V s 3 SOlUti ° n is ™ teims * area. Two 

ZTZr ^ ^ to dlustrate ** frade off between redundancy, ergonomics of 
use and visible display. Firstly, a full screen keyboard offers direS input with no 

ST^' StaBdard SpHt SCreen ke y board amoves redundancy for the sake 

* ^ ltS SmaU SiZC r6SultS 111 1116 need to ™» 311 additional device, 
such as a stylus, as lt becomes difficult to use dextrously with the fingers 

One approach based on the standard keyboard and akin to one I propose is one that 

uses a static soft keyboard placed in the background of the display text A letter is 

selected by tapping the appropriate region in the background. 

This solution permits manual input and does preserve some screen real estate A 

etoHcmt swtn * t0 °" ^ necessitatin S a cumbersome need to 

explicitly switch between numbers, punctuation and other lesser used keys Another 

ttxTMoreov^ T r h r t0 aV ° id ° bSCUring ^ k6yb0ard carefW1 y se * ected ^put 

woul fSSJftJ *?* 311(1 CUck interaCtion is the approach 

would benefit from the addition of gesture interaction to promote a greater degree of 
redundancy and richer level of interaction. ^ 

3.5 Statistically Optimised Single Digit Keyboard 

ST ^ ndedto improve the soft keyboard approach, however 
these attempts are still subject to the drawbacks already describe with this approach 

keX^IlayoSt. 31 * 6 "**"* * * ° VBdMad imp ° Sed by remode ^g the 

^o^nY^,^ keyb0ard Hunt6r 6t 31 (2000 >' M ankoff and Abowd (1998); Zhai et 
Tu Th all 1 let ^ ers 5* ^distant, thus eliminating excessive key homing distances. 
The Metropolis keyboard Zhai et al (2000) is another optimised soft keyboard layout, 
statistically optimised for single finger input, improving efficiency by placing 
frequently used keys near the centre of the keyboard. . 
Both approaches can be effective, but both impose a learning overhead due to a new 
keyboard layout. The user must expend considerable effort to become familiar with 
the keyboard for relatively slim rewards, not to mention the overhead inherent with 
soft keyboards, such as the consumption of screen real estate 



3.6 Handwriting Recognition 



—u UW , , lu ,g Recognition 

slower, some 25wpm at best, CS^rT"? ** *** is bal ^ ^ 
such as the soft keyboard Dix eta 1(1^1) ' 7 ' ^ leSS S0 P*sticated approaches, 

The obvious problem with hanH™^ • 

word, etc., w4easL^eyb^S 1 o lnPUt ? Ueed t0 «* letter of each 
addition to this ^mc^Z ^T^^T^ *» pieSS ^ of a ^£ 
ft. use of a stylus, to ooc^^fl^SS £ h^T*' *" *** r ^s • 
PDA) when entering text. S S6r S free hand (>•«•. the need to hold the 

3.7 Alternative Approaches 

3.8 Virtual Keyboards 

2SSSssi£ 2^^: VST 8 of a ^ ***** ™ s — 

keyboard. w me * ser to type on any surface as if it were a 

h^ds^ record biometric data from the 

use any surface as a ^ylZZ T^ZT^"* 0 P e -tor to 

ChmchoUe(1999). A SmuIar a PP r oach is outlined in Goldstein and 

Sensors in the hand units measure th~ r, 

techniques and a language T^str iKS - ^ while ^ial intelUgence 
movements. This novel approachTs Z frS^T appr ° pnate ke ^okes or mouse 
need to carry around two ^ttens ^ 1116 main P r °blem is the 

contribute to the awWdnesHf^nSv tS, ^ ^ ^ PDA itself > «* which 
mutual keyboard must be provided iT^e Tfr * T™* re P rese *tation of the. 
screen area for alternative uses. ' S ° lutl0n does not effectively release. 

3.9 Dynamic Dialogues 

Dynamic dialogues illustrated in Ward et al f2nnm 

interface incorporating language n^a^lm./7£° v ° 0) , •** data entry 

dimensional gestures, wher f^Tu™sS^f s 2j V, J™ 0 by c ^tmuous two! 
the screen. Letters with a higher probaSut oS^^f S *"* pi °& ess acr °ss 
close to the centre line. Probability of being found in a word are positioned 




making them readily learnable. 

P^^t:^TJoit r S ^7^' is — when 

W^-ala^^tag,^^^^ of 40-60 words per minute. 

more than with a soft keyboard " Up * e ^put dialogue, even 
4 Evaluation of Handheld Text lnput 

- SrS^TS r « of investigation into the 
mputting text per se, but ratheTSe SnsSf ^ iS n0t *" danism for 

— the ftee display region *- 



dynamic input panel would msnlfT,^ 8 T^ 6 Same S5% of * e ««" ■» the 

soft keyboard, which would 
I next discuss the reauir^m^te # j respect to Fitt s law Sears et al (1993). 
for handheld devices " ' COnstraints on > &e design of text input devices 



4.1 Ease of use 



—ww WJ U»C 

Seldom used commands taSMSuS^SSj !T IT* 1 '° menUS ' 
slow, and cumbersome l^-Jtof^SS, flSSS *° " 



4.2 Unnecessary Interaction Aids. 

To ScT^ffiett ^ e d -ice usabili* 

Pick up a stylus, v^L^^^ d °» £ 6 0r > sa * 

Goldstein and Chincholle 0999) * P 6 deViC6S ' tieS U P both han <k 

Ma£ earnin S[li nd Skl " ACqUiS, ' ti0n ° v erhead 

mSLSc and" SoukorffT a^C^ • ^ ^ learned > consi <^ * 
alphanun.eric gestures ^'2^^^^^ - * *> 

5 Design Heuristics 

SuSnT ct bltonsu^lrS * ™» <* the design 

expedient solution, raurer mafSer oZSf *" dCTelo Pr nt ° f a fesh <™> 

dtp.t Sea 2 g °° d b6tWeen redunda -y - input device features and availability of 

a S^tfiCSS StSSs " between display - - 

• It should be easy to learn to use the techniques (MacKenzie and Soukoreff, 1999). 

^^iMSia 1 now *r myproposed ~ » *■ • 

6 Animated Transparent Layers with Gesture Interaction 




v22£^SS of scheme described * H ™ 

to optimize this compromise ( } mvestl 8 atlon of levels of transparency 

illustrated m Beige eTal a£$ffSZ?i K ddmes / uch 35 d ^ic signature!, 
Long et al (1997) to aUow & e 'if ^ t ^ UgUm (1 " 5) ' Joh ^son (1973) and 
simultaneously, a sort 7 Sten^ve fa^inf t ^ ^ ° r P ° SSibly more visua * 
facilitation of a compromise ^ S ° f SCreen real estate ***** than the 

onnn COiPOrati ,° n ° f SimpIe S estures > ^plored in Long et al H997^ ^ p u 
(2001), is an elegant solution that avoir) sth* « *v . ( 997) 311(1 Rak °wski 

more elaborate gestures while offeri^H v T 31186 in the cognition of 
restricted point aad click 21? addltlona l context required beyond that of the 

implementing oteXXs' ^r^ 8 ^* 1 iS P&rtly a ^quence of 

interaction. To avoW thl over^L^ f necessai y to ^solve issues of layer 
to address, for Ix^ e S 

interaction is used to ^Kl^jSS^ " " a ^ ™ 
7 Proposed Solution 

^vices uses the visual overloading 
(1992), Cox et al aSxSTZF^^^P^ 38 describ ^ m Bartlett 

dynamic signatures as demo^JS^ £ T Y^T (1 " 6) ^ addi ^ of 
and SuguruTl995) and IZers Tsls) ^ *? ( ?" 3) i * 31 (1 " 4 >> Ishantha 
superimposing of layers of Sou l 1o ^ <* animated transparencies permits the 
to the coherency of c^Syt S?^! C ° mprom ise 

between display area and 'in™,? 2l ( ?" )- ? Way ' 311 arable balance. 
al(1996). ^ dmpUtdiai °^ ecanbe Wished, as discussed mK^^ 

Fig. 1. A tnock up of the proposed solution', a superposed Skipnun, keyboard on a .ayer of text^ 

a^me^ £ — »*■ t Veiy ^ — ~ 

interaction, as in Long et^ 0997) T^sSect a t ^ m ° re 

starts over the selected button ^is7erfomS ?he • ^ ** ends or 

keyboard elements assists m afford^T J ^ff ° m ° tl0n ° r of the 

permits a simple solution to aSSnf Si T?* ° f ^ g6StUre - ^ use 6f ^stare 
outlay of extra disp?^e a TcZL?, With ° Ut tile overhead * *e 

Rakowski (2001) X ex^nle th^r T? ^t"? ^P****. as discussed in 
of letters, numbers SlSita ZuvlTn * POP ^ d layers 

a letter and a downward ee^ tw, P Z™*™ 1 ffaduaA * esture would select 
selects me approp^^^ 

activate a command that displavs a " » thi* lu u •!? 0uld be used to 

screen or anywhere, making T^oL con^and * * "** ^ ° f *" 

2* -PP^ a gradient stroke that ends over the button. T*e 
^n^ture^ wouid be se.ected with a ,e ft 

somewhat more coherent y erotletters - wnen "> motion, stand out against the background and appears 
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8 Evaluation 

The above approach leads to several benefits, as now discussed. 

• There is no reliance on additional interaction aids, as the dialogue elements are 
large enough to support manual operation. 

• The keyboard is somewhat less awkward to use, since it is large enough to 
interact without using styli, and so on, without compromising redundancy or 
screen real estate. 

• The approach reflects an effective combination of redundancy in input device 
features and availability of display area. 

• Finally, the approach provides an adequate trade-off between display area, size 
of elements m the input panel, and usability. 

As has been illustrated, my visual overloading technique solves the text entry problem 
without imposing significant overhead. Moreover, with appropriate training, a user 
can mput text at rates comparable to that of standard soft keyboards (i.e., around 
40wpm). What we lose by having to make a gesture is gained through having a larger 
area to select (cf Fitt's law). Moreover, the gesture solution implemented is simple, 
and is relatively error free, and does not suffer from the computational overhead of 
the more elaborate gesture recognition approaches. 

9 Conclusion 

This paper has illustrated the constraints on and issues relating to, the development of 
text input devices for mobile and wearable devices, as illustrated in Dunlop and 
Crossan (1999) and MacKenzie and Soukoreff (1999). A solution to the problems and 
shortcomings of existing schemes was proposed. A prototype system, making use of 
gestures and visual overloading, was also described. It was argued that this prototype 
makes effective use of screen area, preserves the portability of the device, and does 
not suffer from the computational overheads of more complex gestural interfaces 
My current work involves investigating the application of my techniques to support 
text input for Databoards, public information Kiosks (considered by Christian and 
^ 6ry - .lll^r ^-^S™ ™P ut for very small devices, such as wearable devices 
(Masui, 1999). I are exploring the effectiveness of visual overloading itself and 
seeking to improve touch screen interaction, among other things. I also intend to 
explore the use of techniques in a predictive text application. 

Finally, I recognise that our future research will benefit from an investigation into 
•theories of perception. Such work may help us to niinimise, and govern the effects of 
visual rivalry, perhaps by introducing 3D elements, as discussed in Pourang (2000a 
and 2000b), and dynamic shading and elements, studied by Bier et al (1994) and 
McGuffin and Balakrishnan (2002). ■ 
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Abstract 
Visually Overloaded Keyboard 



Text input with a PDA is no. as easy as it should be, especially when compared ,o a 
desktop set up win, a standard keyboard. The plethora of attempted solutions ,o the 
text input problem for mobile devices provides evidence of the difficulties, and 
suggests the need for more imaginative approaches. We propose an approach mat 
optimises on the ability of humans to effortlessly distinguish between animated 
transparent overlays. Our approach permits the intensive population of a display 
through the layering of control elements. Moreover we can limit population of these 
controls by overloading their functionality with gesture interaction. This we achieve 
without compromise in size of tire inputted text pane or to to size of control 



elements. 




Direction of 
rotation of- 
overlaid 
dialogue 



1/2 



Overlaid alphanumeric input 
dialogue 
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Background or work context 



Ia?er o W* UP ° f Pr ° P ° Sed S ° luti ° n1 ' a superimposed Skipnum keyboard 



on a 



■ SKIER: tSS or S; h ZT \ T m muItip,e ,ayere - of u > to £ ^ AAskl 

somewhat more coherent layer or letters, when in motion, stand out against the background and appears 




tftmitri&tuig gesture, while -<sp.ee>" would be Reeled wil « downwart Z! * 
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